[Fix] Billed API requests return no response when streams fail silently or end at max_tokens - #1580

Summary by CodeRabbit

• New Features
  • Added bounded automatic retries for empty responses and mid-stream failures.
  • Retry progress is displayed, with user prompts after automatic retries are exhausted.
  • Retry handling avoids duplicate conversation messages and preserves message identity.
  • Token-limit terminations now provide a specific error instead of being retried.
  • Preserved signed Anthropic reasoning content across tool-use continuations, including multiple reasoning blocks.
  • Exposed model stop reasons in streaming usage updates.

Walkthrough

The change captures Anthropic thinking signatures and stop reasons, replays signed thinking blocks, adds bounded retry handling, preserves user-message identity, and validates retry state transitions.

Suggested reviewers: hannesrudolph, edelauna, navedmerchant

Changes

Anthropic streaming and task recovery

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

Anthropic streaming and retry flow

sequenceDiagram
  participant AnthropicAPI
  participant AnthropicHandler
  participant Task
  participant User
  AnthropicAPI->>AnthropicHandler: Stream thinking, signatures, and stop reason
  AnthropicHandler->>Task: Emit completed blocks and usage stopReason
  Task->>Task: Track failures and automatic retries
  Task->>User: Request retry after retry limit
  User->>Task: Approve or decline recovery

Explanation

Focused coverage is incomplete for two changed negative paths. In src/api/providers/anthropic.ts, the new signature_delta accumulation is not guarded by chunk.index === thinkingBlockIndex, but the tests cover only mismatched thinking_delta and mismatched content_block_stop; no test sends a mismatched signature_delta and verifies that it cannot contaminate a signed block. In src/core/task/Task.ts, the new mid-stream exhaustion path explicitly avoids removing history when currentUserContent is empty, but all new mid-stream retry tests use a non-empty initial user request. No focused Task test covers an empty continuation with existing history, where an incorrect pop would lose a prior turn. No changed UI component requires a Playwright snapshot.

Resolution

Add an Anthropic provider test with an open thinking block at one index and a signature_delta at another index. Assert that the stray signature is ignored and that the completed block has only the correct signature. Update the implementation to apply the same index guard to signature accumulation if the test exposes the current behavior. Add a Task-level mid-stream failure test for an empty continuation with pre-existing history. Drive it through retry exhaustion and assert that the existing history and message counts remain intact, with the expected user prompt and failure recording.

Explanation

The changed mid-stream retry path bypasses the global approval control. In the base code, src/core/task/Task.ts gated backoffAndAnnounce and the automatic retry on stateForBackoff?.autoApprovalEnabled. The pull request removes that gate and retries up to three times whenever the stream fails (midStreamRetryAttempt < MAX_AUTOMATIC_API_RETRIES). The new test explicitly confirms this behavior with autoApprovalEnabled: false. Therefore, when a user disables auto-approval and an API stream fails after partial output, the task sends three additional API requests before presenting api_req_failed; those requests occur without user approval.

Resolution

Keep automatic mid-stream retries behind the autoApprovalEnabled control, or require an explicit api_req_failed approval before the first retry when auto-approval is disabled. Preserve the retry cap and visible countdown after approval, and add a regression test that verifies zero automatic retries when autoApprovalEnabled is false.

Explanation

The new terminal retry paths do not handle a failed persistence result. restoreApiHistoryUserMessage() pushes the removed user record and awaits saveApiConversationHistory(), but discards its boolean result. The new mid-stream failure path then calls addToApiConversationHistory() for the synthetic assistant failure and returns false; that method also continues when saveApiConversationHistory() returns false. saveApiConversationHistory() catches write errors and explicitly returns false. If the API-history write fails because the task directory is unavailable, the disk is full, or the file lock cannot be acquired, the task can stop with the user-visible failure while the restored user record or the new synthetic assistant failure is missing from persisted history. The in-memory history and persisted history can then diverge after restart. The same unchecked restore result is used in the new max_tokens and empty-response terminal paths.

Resolution

Propagate persistence success from the new restore and terminal-recording paths. If restoring the user message fails, retry with the existing bounded persistence retry mechanism and do not continue or report a terminal result until the history is durable. If the synthetic assistant failure cannot be persisted, apply explicit partial-failure handling: retry, report the persistence failure, and prevent task completion, or roll back the in-memory mutation. Add tests that make saveApiConversationHistory() fail during restore and during synthetic failure recording, then verify that the task does not finish with divergent in-memory and on-disk history.

Explanation

The new mid-stream retry-exhaustion path adds an api_req_failed ask at src/core/task/Task.ts:3724-3742. api_req_failed is an idle ask, so ask() schedules a 2-second status timer at src/core/task/Task.ts:1634-1644. If the task is cancelled while this ask waits, the abort branch at src/core/task/Task.ts:1673-1679 throws before the normal cleanup at src/core/task/Task.ts:1701-1702; dispose() does not clear these local timers (src/core/task/Task.ts:2729-2751). The timer then runs after cancellation, retains the task until it fires, and can set idleAsk and emit TaskIdle. Triggering four mid-stream failures and cancelling during the new prompt reproduces this changed lifecycle path.

Resolution

Make ask-owned status timers cancellation-safe. Clear the local timeouts before the abort and superseded exits, or store them as task-owned handles and clear them from abortTask() and dispose(). Ensure the timer callback also checks this.abort or disposal state before mutating task state or emitting events.

Explanation

The description explains the changes, implementation, tests, scope, and documentation impact. However, the required approved GitHub Issue is missing; the description states that the report exists only in Discord and leaves the issue reference unresolved.

• X
• Mastodon
• Reddit
• LinkedIn

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.